<html>
<head><meta charset="utf-8"><title>Semantic Token delay · t-compiler/rust-analyzer · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/index.html">t-compiler/rust-analyzer</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Semantic.20Token.20delay.html">Semantic Token delay</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="234582261"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Semantic%20Token%20delay/near/234582261" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Domenic Quirl <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Semantic.20Token.20delay.html#234582261">(Apr 14 2021 at 21:50)</a>:</h4>
<p>Hi everyone. I hope it's fine to pop in here without working on r-a (I asked Borger about it and he said it'd be fine, feel free to tell me if it's not). For those who don't know me, I briefly appeared previously in <a href="#narrow/stream/185405-t-compiler.2Frust-analyzer/topic/PSA.3A.20rowan.20might.20be.20a.20severe.20case.20of.20over.20engineering">https://rust-lang.zulipchat.com/#narrow/stream/185405-t-compiler.2Frust-analyzer/topic/PSA.3A.20rowan.20might.20be.20a.20severe.20case.20of.20over.20engineering</a> as someone who made <span class="user-mention" data-user-id="133169">@matklad</span> think about <code>rowan</code> even more than he already did.</p>
<p>Recently, I started writing a language server for the thing I also use <code>cstree</code> for, and the other day I ran into <a href="https://github.com/microsoft/vscode/issues/89640">https://github.com/microsoft/vscode/issues/89640</a> : because of using semantic highlighting, and the LSP request for the token taking some small amount of time, there is a noticeable delay until highlighting updates, for example when you comment out a line. This has since started to annoy me tremendously (and probably more than it really should), because</p>
<p>- changes like commenting out a single line are so small and "simple" that it just _feels_ like the update should be instant, and<br>
  - the entire file is _syntactically_ highlighted basically on the spot upon opening it from the textMate language grammar.</p>
<p>The delay scales with file length, but I can see it even in small files (r-a does slightly better than I do currently for these sizes, though I'm unsure why exactly). So I'm looking for a way to have some visual update happen more quickly, through either the language server or the extension. The latter side I am criminally unfamiliar with and have just cobbled together enough to make the whole thing work. </p>
<p>Have you guys implemented anything specifically to help with this for r-a, or looked at this previously, or does r-a do something more general to speed up turnaround times on "important" updates? I do realize this is a minor issue in the grand scheme of things and I am kinda obsessing over this a bit, but such obsessing is a strong suite of mine and I'm taking a longer shot to try to convert that into something useful <span aria-label="sweat smile" class="emoji emoji-1f605" role="img" title="sweat smile">:sweat_smile:</span></p>



<a name="234582474"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Semantic%20Token%20delay/near/234582474" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Domenic Quirl <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Semantic.20Token.20delay.html#234582474">(Apr 14 2021 at 21:51)</a>:</h4>
<p>Again, I don't want to disrupt your work-related Zulipping ('s that a word?), so do tell me if I do</p>



<a name="234583632"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Semantic%20Token%20delay/near/234583632" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Lukas Wirth <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Semantic.20Token.20delay.html#234583632">(Apr 14 2021 at 22:01)</a>:</h4>
<p>Asking this on zulip is completely fine.<br>
The same delayed highlighting behavior appears in RA when commenting out a line, at least for me and I don't think that came ever up as an issue somewhere.<br>
The only thing I can think of that might reduce the time a bit is handling delta highlighting via <code>textDocument/semanticTokens/full/delta</code> but as RA also already does that I don't believe much can be gained from that.</p>



<a name="234583768"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Semantic%20Token%20delay/near/234583768" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Lukas Wirth <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Semantic.20Token.20delay.html#234583768">(Apr 14 2021 at 22:02)</a>:</h4>
<p>Though I don't have extensive knowledge about the spec or implementations either</p>



<a name="234585211"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Semantic%20Token%20delay/near/234585211" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Domenic Quirl <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Semantic.20Token.20delay.html#234585211">(Apr 14 2021 at 22:14)</a>:</h4>
<p>Yeah, I did implement token deltas, but in a way which is close to the VSCode typescript implementation (if I'm not mistaken RA's implementation is also similar). So the delta request will recompute all file tokens anyways, and only save on the tokens sent to the editor.</p>



<a name="234585482"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Semantic%20Token%20delay/near/234585482" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Domenic Quirl <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Semantic.20Token.20delay.html#234585482">(Apr 14 2021 at 22:17)</a>:</h4>
<p>So far I've noticed RA to be a bit faster for the smaller files, while I catch up as files sizes increase. But that comparison is in no way fair, since for one RA does a lot more other work, and also I believe you use a sync base + thread pool (?) while I set up with <code>lspower</code> for now, which is async</p>



<a name="234625098"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Semantic%20Token%20delay/near/234625098" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Semantic.20Token.20delay.html#234625098">(Apr 15 2021 at 06:49)</a>:</h4>
<p>We didn't do anything specifically to improve hightlightting latency, but let me dump general facts I know</p>



<a name="234625121"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Semantic%20Token%20delay/near/234625121" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Semantic.20Token.20delay.html#234625121">(Apr 15 2021 at 06:49)</a>:</h4>
<p>First, LSP spec is not really suitable for fast and precise coloring.</p>



<a name="234625272"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Semantic%20Token%20delay/near/234625272" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Semantic.20Token.20delay.html#234625272">(Apr 15 2021 at 06:50)</a>:</h4>
<p>For precise coloring, you need to use type-inference results, and, in general, type inference can be slow, so this part needs to be async in terms of user experience (ie, when opening a file, the users will see more correct syntax highlighting kicking in milliseconds later)</p>



<a name="234625426"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Semantic%20Token%20delay/near/234625426" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Semantic.20Token.20delay.html#234625426">(Apr 15 2021 at 06:52)</a>:</h4>
<p>What you can real-time guarantees for is lexer-based hightlighting. Lexing is done using finite state machines, and, unlike parsing or type-checking, it's easy to re-start the lexer from a specific point, so you can re-lex incrementally in  bounded time.</p>



<a name="234625529"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Semantic%20Token%20delay/near/234625529" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Semantic.20Token.20delay.html#234625529">(Apr 15 2021 at 06:54)</a>:</h4>
<p>So, the proper API for syntax hightlghting (like the one in IntelliJ obviously) would have <em>two</em> methods, one for synchroneous lexer-based highlighting (so that you can compute  the correct colors before the next display flip) and one for asynchronouse type-inference based hightlighting.</p>



<a name="234625731"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Semantic%20Token%20delay/near/234625731" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Semantic.20Token.20delay.html#234625731">(Apr 15 2021 at 06:56)</a>:</h4>
<p>The other thing which is important for latency is proper management of read/write requests.</p>



<a name="234625847"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Semantic%20Token%20delay/near/234625847" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Semantic.20Token.20delay.html#234625847">(Apr 15 2021 at 06:57)</a>:</h4>
<p>Let's say user typed <code>foobar</code>. What your server might see in the protocol is</p>
<ul>
<li>insert <code>"foo"</code> into main. rs</li>
<li>hightlight <a href="http://main.rs">main.rs</a></li>
<li>insert <code>"bar"</code> into <a href="http://main.rs">main.rs</a></li>
<li>hightlight <a href="http://main.rs">main.rs</a></li>
</ul>



<a name="234627307"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Semantic%20Token%20delay/near/234627307" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Semantic.20Token.20delay.html#234627307">(Apr 15 2021 at 07:05)</a>:</h4>
<p>in this situation, it's important to make sure  that you can react to insert <code>bar</code> immediately, without waiting for hightlighting to finish. </p>
<p>The approach we use for this in rust-analyzer is that we have a manually-coded event loop which accepts the messages. With this loop, we can schedule latency-critical operations (all edits, but some of the read operations as well) onto the loop's thread directly, and offload async operations to the background ("should this OP be synchroneous/block the main loop?" is a useful framing question). See <a href="https://github.com/rust-analyzer/rust-analyzer/blob/e131bfc747df1b21ae6ea04eb9c55001e06b7bf0/crates/rust-analyzer/src/main_loop.rs#L494-L515">https://github.com/rust-analyzer/rust-analyzer/blob/e131bfc747df1b21ae6ea04eb9c55001e06b7bf0/crates/rust-analyzer/src/main_loop.rs#L494-L515</a> <code>on_sync</code> and <code>on</code> are the two flavors. </p>
<p>Separately from scheduling, you need "async op can't block a sync one" property. I think there are two solutions here: either you make the state to be a purely functional data structure, so that async and sync can execute on two different versions of state at the same time, or you add cancellation for async operations, so that you can cancel pending jobs.</p>



<a name="234627563"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Semantic%20Token%20delay/near/234627563" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Semantic.20Token.20delay.html#234627563">(Apr 15 2021 at 07:07)</a>:</h4>
<blockquote>
<p>you use a sync base + thread pool (?) while I set up with lspower for now, which is async</p>
</blockquote>
<p>I am in general async-sceptic, so I might be biased, but I don't think per se async buys anything for LSP server use-case :)</p>



<a name="234642581"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Semantic%20Token%20delay/near/234642581" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Domenic Quirl <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Semantic.20Token.20delay.html#234642581">(Apr 15 2021 at 09:18)</a>:</h4>
<p>Thank you! I very much appreciate the dump!</p>



<a name="234645328"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Semantic%20Token%20delay/near/234645328" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Domenic Quirl <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Semantic.20Token.20delay.html#234645328">(Apr 15 2021 at 09:39)</a>:</h4>
<p><span class="user-mention silent" data-user-id="133169">matklad</span> <a href="#narrow/stream/185405-t-compiler.2Frust-analyzer/topic/Semantic.20Token.20delay/near/234625529">said</a>:</p>
<blockquote>
<p>So, the proper API for syntax hightlghting (like the one in IntelliJ obviously) would have <em>two</em> methods, one for synchroneous lexer-based highlighting (so that you can compute  the correct colors before the next display flip) and one for asynchronouse type-inference based hightlighting.</p>
</blockquote>
<p>I am inclined to agree, hence me trying to find a way to react to such changes more quickly with a coarser approximation, while full results are pending. Based on what all of you said here so far, it seems to me like if there is a way to make this work, that way is likely it would involve some custom code in the editor extension. Is there an (or a combination of) API access point(s) that one could use to trigger re-highlighting a part of a document with the textMate grammar, or maybe just remove the semantic highlighting scopes? If I understand correctly they just get put "on top" of the textMate ones, so if they are gone highlighting would fall back to syntactic?</p>



<a name="234645573"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Semantic%20Token%20delay/near/234645573" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Semantic.20Token.20delay.html#234645573">(Apr 15 2021 at 09:41)</a>:</h4>
<p><a href="https://code.visualstudio.com/api/references/vscode-api#DocumentRangeSemanticTokensProvider">https://code.visualstudio.com/api/references/vscode-api#DocumentRangeSemanticTokensProvider</a></p>



<a name="234645627"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Semantic%20Token%20delay/near/234645627" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Domenic Quirl <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Semantic.20Token.20delay.html#234645627">(Apr 15 2021 at 09:41)</a>:</h4>
<p>As I said I'm still getting into the TS side of things. But it appears to be possible to register for additional notifications in the extension for things like <code>DocumentChangeEvent</code>s. So you could know those and check if they represent a change for which you want "intermediate" highlighting. But you'd have to be able to trigger that then happening</p>



<a name="234645629"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Semantic%20Token%20delay/near/234645629" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Semantic.20Token.20delay.html#234645629">(Apr 15 2021 at 09:41)</a>:</h4>
<p>it has <code>didChangeSemanticTokens</code>, so it should be possible to hack something working here</p>



<a name="234645831"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Semantic%20Token%20delay/near/234645831" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Domenic Quirl <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Semantic.20Token.20delay.html#234645831">(Apr 15 2021 at 09:43)</a>:</h4>
<p>(your other points are also very helpful, but are something I will deal with later)</p>



<a name="234646070"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Semantic%20Token%20delay/near/234646070" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Semantic.20Token.20delay.html#234646070">(Apr 15 2021 at 09:45)</a>:</h4>
<p>But, in terms of system engineering, I'd personally would:</p>
<ul>
<li><span aria-label="shrug" class="emoji emoji-1f937" role="img" title="shrug">:shrug:</span> at the issue and shake the fist in the general direction of broken software</li>
<li>made sure that <em>internals</em> of the thing I am writing are prefect, such that, if anyone comes up with a better API than VS Code, I could just plug this in</li>
<li>raise the issue in the VS Code repository, which describes the situation like "hey, we <em>could</em> have a perfect UX here, look how my server does this, but we currently cant, because of the clients limitations". VS Code folks are moderately responsive -- it takes some years to move the status quo forward, but today's API are much better that what we've started with in 2018. Like, we <strong>have</strong> hightlighting at all :-)</li>
</ul>



<a name="234646125"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Semantic%20Token%20delay/near/234646125" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Domenic Quirl <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Semantic.20Token.20delay.html#234646125">(Apr 15 2021 at 09:45)</a>:</h4>
<p><span class="user-mention silent" data-user-id="133169">matklad</span> <a href="#narrow/stream/185405-t-compiler.2Frust-analyzer/topic/Semantic.20Token.20delay/near/234627563">said</a>:</p>
<blockquote>
<p>I am in general async-sceptic, so I might be biased, but I don't think per se async buys anything for LSP server use-case :)</p>
</blockquote>
<p>Though I will say I am also unsure about this, since a lot of the LSP operations do in some way rely on each other, and you are not that much bound by awaiting something longer like IO. I primarily felt this was very easy to get going</p>



<a name="234647025"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Semantic%20Token%20delay/near/234647025" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Domenic Quirl <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Semantic.20Token.20delay.html#234647025">(Apr 15 2021 at 09:53)</a>:</h4>
<p><span class="user-mention silent" data-user-id="133169">matklad</span> <a href="#narrow/stream/185405-t-compiler.2Frust-analyzer/topic/Semantic.20Token.20delay/near/234645629">said</a>:</p>
<blockquote>
<p>it has <code>didChangeSemanticTokens</code>, so it should be possible to hack something working here</p>
</blockquote>
<p>Hm, so you're thinking something like</p>
<ul>
<li>register a token provider (I think in the VSC case <code>Range</code> is only used once when opening the file to initialize the viewport, and then everything else is delta requests) that mainly forwards to the lsp</li>
<li>register for document change events, if it's something where I want to change highlighting, store that somewhere and <code>didChangeSemanticTokens</code> from the provider</li>
<li>if provider is called with some "override" info present, provide an edit with the intermediate change<br>
?</li>
</ul>



<a name="234647479"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Semantic%20Token%20delay/near/234647479" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Domenic Quirl <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Semantic.20Token.20delay.html#234647479">(Apr 15 2021 at 09:56)</a>:</h4>
<p>Would the editor not issue a request for a token update anyways because the file is updated? Also does the TS method correspond to <code>semanticTokens/refresh</code> in terms of the LSP spec? Cause then one would probably need to provide a full set of tokens <span aria-label="thinking" class="emoji emoji-1f914" role="img" title="thinking">:thinking:</span></p>



<a name="234647509"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Semantic%20Token%20delay/near/234647509" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Domenic Quirl <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Semantic.20Token.20delay.html#234647509">(Apr 15 2021 at 09:56)</a>:</h4>
<blockquote>
<p><span aria-label="shrug" class="emoji emoji-1f937" role="img" title="shrug">:shrug:</span> at the issue and shake the fist in the general direction of broken software</p>
</blockquote>
<p>this <em>does</em> seem to be at least part of the way to go ^^</p>



<a name="234648900"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Semantic%20Token%20delay/near/234648900" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Domenic Quirl <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Semantic.20Token.20delay.html#234648900">(Apr 15 2021 at 10:07)</a>:</h4>
<p>I was hoping there was some <a href="https://code.visualstudio.com/api/references/commands">https://code.visualstudio.com/api/references/commands</a> that allows to interact with VSC on the topic of highlighting, but it seems you can only ask it to ask the LS about something for you</p>



<a name="234696181"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Semantic%20Token%20delay/near/234696181" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jeremy Kolb <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Semantic.20Token.20delay.html#234696181">(Apr 15 2021 at 15:16)</a>:</h4>
<p><span class="user-mention silent" data-user-id="337115">Domenic Quirl</span> <a href="#narrow/stream/185405-t-compiler.2Frust-analyzer/topic/Semantic.20Token.20delay/near/234647479">said</a>:</p>
<blockquote>
<p>Would the editor not issue a request for a token update anyways because the file is updated? Also does the TS method correspond to <code>semanticTokens/refresh</code> in terms of the LSP spec? Cause then one would probably need to provide a full set of tokens <span aria-label="thinking" class="emoji emoji-1f914" role="img" title="thinking">:thinking:</span></p>
</blockquote>
<p><code>semanticTokens/refresh</code> is sent from the server to the client. We do that on a workspace reload if I remember correctly.</p>



<a name="234696454"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Semantic%20Token%20delay/near/234696454" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jeremy Kolb <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Semantic.20Token.20delay.html#234696454">(Apr 15 2021 at 15:18)</a>:</h4>
<p>In terms of the delta we compute the full set of tokens and just send the delta back but it's a pretty naive algorithm that I took from <code>clangd</code></p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>